Skip to content

Harden reviewer gate, edit recovery, and spawn/tool-call robustness#26

Open
AnzoBenjamin wants to merge 11 commits into
mainfrom
fix/reviewer-gate-hardening
Open

Harden reviewer gate, edit recovery, and spawn/tool-call robustness#26
AnzoBenjamin wants to merge 11 commits into
mainfrom
fix/reviewer-gate-hardening

Conversation

@AnzoBenjamin

@AnzoBenjamin AnzoBenjamin commented Jul 23, 2026

Copy link
Copy Markdown
Owner

Hardens agent-spawn and tool-call validation so common model mistakes fail with actionable guidance or safe auto-repair, routes pure test-coverage reviewer findings to the right specialist, and clarifies the hooks→reviewer gate contract for orchestrators.

Spawn / tool-call hardening

  • Single-agent mis-braced spawn payload auto-repair (multi-agent stays fail-closed)
  • Sharper spawn_agents hint with corrected inline example
  • Concrete code-searcher recovery for ungranted content-search tools
  • base2 spawn-guidance prompt reinforcement; debug logging on the auto-repair path
  • get_build_targets empty-files recovery
  • Cyclic and deep-safe containsStructuralAuditReceipt via min-depth WeakMap tracking
  • Targeted spawn recovery hints for reviewer-family, dependency-manager, and librarian

Reviewer-gate routing (all-coverage → test-writer)

  • Export + inline mirror of isTestCoverageReviewerFinding (handleSteps serialization-safe)
  • When every open reviewer blocker is coverage-only, spawn only test-writer (not repair-editor)
  • Mixed or code-only blocker sets still spawn only repair-editor
  • Gate messages / nextRequiredAction label the active agent correctly

Gate / hooks guidance (four clarifications)

  • Re-arm ownership: runtime-owned path is run_file_change_hooks (model-hidden, injected) → automated code-reviewer → finalization; wait when awaiting_validation
  • run_targeted_validation is NOT the gate: scoped evidence only; does not clear findings or unlock git-committer (same for basher typechecks)
  • Commit recovery: gate re-arms on every edit; treat "not available yet" as normal ordering; spawn committer once after green with owned_paths
  • Pending-set authority: full pendingGateFiles set is authoritative over conversational memory
  • Docs + base2/base-deep tool-choice/validation bullets aligned; deep Phase 6 Final Review before Phase 7 Lessons

Tests

  • Full agents/__tests__/base2.test.ts suite; gate-reviewer unit + parity; quality-prompt-snapshot topic guards
  • Agents typecheck green

This change is Reviewable

Extract containsStructuralAuditReceipt into @codebuff/common/util/audit-receipt and rewire both the general-agent handleSteps completion gate and spawn-agent-utils buildRuntimeAgentReceipt gate to the shared helper so the two receipt checks cannot drift; cover the helper with a unit test.

- general-agent prompt now routes ripgrep-style search through code-searcher (code_search is a registered runtime tool but intentionally not granted to general-agent) and names the required params.searchQueries.

- add a code-searcher searchQueries recovery hint in validateAgentInput so an empty spawn gets an actionable message instead of a bare schema error.

- add general-agent audit-loop tests covering the receipt-present break and the retry-exhaustion break.

- fix the base2 reviewer-spawn coverage-evidence wording from 'below' to 'above' to match where the list is rendered.

All changes pass typecheck and the affected test suites.
Tighten the reviewer gate and edit-application paths so authorization and review coverage cannot be silently bypassed.

Skip the validation/reviewer gate in PLAN-only mode, where no source edits exist to attest to.

Make co-changed tests first-class reviewable files so reviewer attestation covers them instead of always reporting coverage as missing/uncertain; keep them separately identifiable as coverage evidence.

Recapture the review snapshot after validation (validation-first), since hooks can mutate generated or formatted output before review.

Preserve valid read authorization on deterministic edit_transaction and str_replace preflight failures, and only require a fresh whole-transaction re-read when a capability is genuinely stale or out of scope, so a real freshness failure cannot be sidestepped by refreshing one path while replaying stale tokens for the rest.

Scope the edit-application coordinator's positive-evidence confirmation to paths that actually produced a client change, so a no-op path no longer forces a false rejection.

Block replace_range in a transaction once a prior non-range edit has shifted the file's line count, preventing it from splicing the wrong lines while its original-snapshot hash check still passes.

Broaden spawn_agents validation diagnostics with canonical recovery hints for the reviewer family, dependency-manager, librarian, and get_build_targets, and harden audit-receipt traversal against deep/cyclic structures.

Adds regression tests covering each behavior.
Detect a top-level spawn_agents payload where prompt/params/handoff appear as siblings of the agents array, and return the base spawn_agents guidance before the non-strict schema silently drops the stray key.
@vercel

vercel Bot commented Jul 23, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
openbuff Ready Ready Preview, Comment, Open in v0 Jul 23, 2026 10:21pm

Align two base2 reviewer-spawn e2e prompts to include a code-intent keyword so proactive query_index fires as the first yield. Update the rewrite-symbol client-rejection assertion to match the branch's deterministic-rejection semantics (rejection preserves read authorization). Add a base2 test pinning which prompts trigger proactive query_index vs skip to git_status.
… cap

Track the shallowest depth each object was visited at (WeakMap instead of WeakSet) so a shared receipt first reached beyond MAX_TRAVERSAL_DEPTH via a deep path is still found through a later shorter path; equal-or-deeper revisits and cycles are still skipped. Adds a regression test.
Fold a stray top-level prompt/params/handoff sibling into a single-agent spawn_agents array when unambiguous (multi-agent and unexpected-sibling cases still fail closed); add a corrected-shape example to the spawn_agents validation hint; emit a concrete copyable code-searcher spawn recovery for ungranted code_search/find_files_matching_content instead of the generic not-granted line (message-only, still fail-closed); and reinforce the base2 spawn guidance with the correct spawn_agents shape. Adds focused tests.
Reproduces the real-world failure where the whole agents value is a JSON string with a stray prompt pair floating as a sibling array element; asserts it fails closed with corrected-shape guidance.
@AnzoBenjamin AnzoBenjamin changed the title Fix/reviewer gate hardening Harden reviewer gate, edit recovery, and spawn/tool-call robustness Jul 23, 2026
Add a debug log on the single-agent mis-braced spawn_agents auto-repair path, recording only the folded sibling key names and toolCallId so prompt/params/handoff values are never leaked. Thread an optional logger through parseRawToolCall and detectMisbracedSpawnPayload from executeToolCall to enable it.

Add a processStream integration test asserting the folded repair reaches the spawn handler: the repaired payload emits a spawn_agents tool_call and produces no error event.
The two git-committer block messages in tool-executor now describe the validation/reviewer gate re-arming on every edit and clearing automatically, framing the block as expected ordering rather than a failure so agents wait one gate cycle instead of retrying in a loop.

gitDisciplineSection carries matching wait-and-commit guidance, and the run-agent-step-tools and quality-prompt-snapshot tests pin the new wording so the behavior-changing messages stay covered.
Export isTestCoverageReviewerFinding and mirror it inline in handleSteps so pure coverage blockers spawn test-writer only; mixed or code findings stay on repair-editor. Covered by unit, parity, and base2 integration tests.
Expand gateAwareness for re-arm ownership, targeted validation != gate, commit recovery, and pending-set authority. Align base2/base-deep tool-choice and validation bullets; document control-plane and request-flow gate boundaries. Move deep Phase 6 Final Review before Phase 7 Lessons, polish typos, and assert specialistRouting in snapshots.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant